1. Field of the Invention
The present invention relates to the processing of moving images. More particularly, it relates to technologies for inserting watermarks into moving images which are distributed as being accumulated and recorded in package type media, that is, moving images whose storage media are a video tape, a CD-ROM, an MO, a DVD, etc. Also, it relates to technologies for inserting and detecting watermarks into and from moving images of distributed network type distribution or broadcast type distribution, that is, moving images which are transmitted by the Internet, digital CATV and digital satellite broadcast. Further, it relates to technologies for copyright administration services for the moving images which are supplied by the above two distribution methods, and technologies for watermark information authentication services by third parties. More concretely, it relates to an insertion system, an insertion method and a storage medium which are applicable to the technologies mentioned above.
The present invention relates also to copy prevention technology concerning copyright protection.
The technical background which has given birth to the present invention is a situation where anybody can duplicate and edit a work by utilizing a personal computer, owing to the spread of digitization, the popularization of the Internet and the extensive uses of CDs, MOs, DVDs etc., and where problems concerning copyrights have become serious more and more. In order to prevent any illicit copy, therefore, the application of electronic watermark technology to moving images has become indispensable.
2. Description of the Related Art
The present invention deals with the protection of the copyright of moving image information which is circulated or distributed in the form of a package medium or which is transmitted in the form of broadcast, communication or the like. Owing to the recent innovation and mergence of techniques whose core is digitization, the moving image information has been extensively circulated, and the duplication and editing thereof have become easy. Accordingly, the protection of the copyright has become an important theme.
The forms in which the moving image information is circulated or transmitted, include the following:
(a) Analog moving image packages: Cinema, video tape, etc.
(b) Digital moving image packages: CD-ROM, MO, DVD, etc.
(c) Analog moving image broadcasts: Ground wave broadcast, cable TV, satellite broadcast, etc.
(d) Digital moving image broadcasts: Digital TV, digital cable TV, digital satellite broadcast, etc.
(e) Digital moving image communications: Internet communications, etc.
All of these technical fields handle moving images as their contents. Recently, the analog form and the digital form of identical content are converted between each other, and the package, broadcast and communication are employed in optional connections with one another. Especially, a digitized moving image can be readily duplicated into a noiseless copy, and it can be readily edited by a personal computer or the like. Therefore, how the copyright is protected is the important theme at present. Concretely, there are such subjects as whether the duplication is permitted or rejected in the first place, and how piracy is prevented.
Mentioned as prior-art techniques relevant to these subjects are the following:
(1) Scheme wherein, using copy restriction codes, the video recording duplication and reproduction display of moving image information stored in a package medium are permitted or barred in the drive of the medium.
(2) Scheme wherein the storage area of a package medium is divided into a data storage area for storing moving image data therein, and a control storage area for storing control information therein, the latter of which is protected hardware-wise, thereby to prevent the moving image data from being altered by general users.
(3) Scheme (such as MPEG) wherein a digital moving image is subjected to data compression so as to store or transmit the compressed data.
(4) Scheme wherein, in transmitting a moving image, it is encrypted in order that only specific allowed recipients may be able to decrypt the transmitted moving image.
(5) Scheme wherein a watermark camouflaged so as to be visually unperceivable (an invisible watermark) is inserted into the data of a moving image so as to transmit or circulate the resulted moving image.
Even with these schemes, however, a satisfactory scheme for the copyright protection has not hitherto been established yet. This is also obvious from xe2x80x9cCall for Proposalsxe2x80x9d issued under the date of Jul. 1, 1997 by the Data Hiding SubGroup (DHSG) of the Copy Protection Technical Working Group (CPTWG). The call for proposals invites public contributions of those proposed desirable schemes for the copyright protection which employ various xe2x80x9cwatermarkxe2x80x9d techniques conforming to the above scheme (5) and which realize the permission and barring of video recording/reproduction in the above scheme (1).
Note: xe2x80x9cCall for Proposalsxe2x80x9d, Data Hiding SubGroup, Copy Protection Technical Working Group, Version 1.0, Jul. 1, 1997 was available on the World Wide Web at www.dvcc.com/dhsg.
The CPTWG is an association of traders concerned with digital video recording and reproduction, and technologies to be standardized in the trade are taken up here. Such a technology represents a present-day technological level in the field concerned. The proposals to the DHSG which is one of the subgroups of the CPTWG will contribute to the technical standard of the field concerned. Concretely, after being studied as a more comprehensive technology by the DTDG which is another subgroup, the proposals are expected to be reflected in the standard of a secure bus termed xe2x80x9cIEEE1394xe2x80x9d.
The present invention complies also with the Call for Proposals. In this regard, the prior art of electronic watermarking will be stated below.
There have hitherto been practiced copyright administration services for still images and music that utilize electronic watermarks. The electronic watermarking technique in the prior art is such that the ID of a copyright holder, the ID of image data, the ID of the recipient of the image data, the ID of a dealer mediating circulation, the ID of a user, etc. are embedded in accordance with the kind of the digital work. As an expedient for the technique, the image is circulated through the path of the Internet, and the ID numbers are open to anybody. By way of example, products xe2x80x9cPictureMarcxe2x80x9d (Digimarc Corporation in the U.S.) and xe2x80x9cFBI Proxe2x80x9d (High Water Signum Ltd. in the U.K.) correspond to the expedient. As another expedient, the image is circulated by utilizing the Internet or the like, and the ID numbers are managed by key data. By way of example, products xe2x80x9cSysCopxe2x80x9d (Fraunhofer Center for Research Computer Graphics in the U.S.), xe2x80x9cTiger Marc Image/Data Bladexe2x80x9d (NEC America in the U.S.) and xe2x80x9cArgentxe2x80x9d (Dice Co. in the U.S.) correspond to this other expedient.
Regarding researches on copyright administration schemes for moving images that utilize electronic watermarks, various methods in which a bandwidth compression technique (MPEG2) for the moving image is combined with the electronic watermarking have been proposed, and they are basically classified into two types. The first of these types is a data hiding scheme wherein electronic watermark information is inserted in a spatial frequency region so as to, spread the watermark information in a broad spectrum region. The second type is a scheme wherein the electronic watermark is superposed on and inserted into sampled value regions of two dimensions or three dimensions. With the first type, the watermark information is difficult to remove, so that the intensity of the watermarking is high, but the image quality or tone quality of the original data degrades conspicuously. Moreover, frequencies need to be converted twice (as conversion and inversion). Therefore, the burden of processing on the first type is heavier than that on the second type wherein the watermark is embedded in the sampled value regions. Accordingly, the first type requires dedicated hardware. The basic technical themes of the electronic watermarking are (1) that the watermark information is embedded with the degradation to the image quality or the tone quality avoided to the utmost, and (2) that the embedded watermark information is made difficult to remove to the utmost.
In the scheme wherein the watermark information is embedded in the frequency components, it is embedded in the high frequency components of those parts of image data or voice data whose changes are inconspicuous to the human eye, for example, the parts of the hair of the head and the contour of an object, in order to suppress the degradation. When the quantity of the watermark information data to be embedded is increased and is spread in the principal components of the original data, the intensity of the electronic watermarking becomes high. However, when the spread of the watermark information is too broad, the image quality or the tone quality degrades. Moreover, the watermark sometimes disappears or deteriorates due to MPEG encoding/decoding.
Also in the scheme wherein the watermark information is embedded in the sampled value regions, it is embedded in redundant parts so as not to be seen or heard as noise. By utilizing the visual characteristics and aural characteristics of man, the quantity of watermark data to be embedded in the redundant parts is made large, and the quantity of the watermark information in the principal parts of the content is made small. Thus, the watermark information is embedded inconspicuously.
There are also different schemes wherein watermark information is embedded by utilizing the existing techniques for image bandwidth compression. By way of example, Fraunhofer CRCG (in U.S.) has developed the scheme wherein, in embedding the watermark information in the process of the compression of the JPEG, that is, at the stage of linear quantization, a medium frequency coefficient among DCT output coefficients is varied, and the variation is used as the watermark information. Besides, NTT (in Japan) or NEC (in Japan) has developed the scheme wherein, when the conversion coefficient of DCT is to be linearly quantized, a coefficient value is changed little by little so as to form the watermark information. In addition, Mitsubishi Denki (in Japan) is developing conjointly with Kyushu University (in Japan), electronic watermarking which employs wavelet conversion. Kyoto Institute of Technology (in Japan) has proposed the scheme wherein the watermark information is embedded in the moving vectors of MPEG. Concretely, a table in which the watermark bits of 100 bits are held in correspondence with the individual moving vectors existent in the number of 330 per frame, is prepared as secret information at each of transmission and reception ends. It is secretly utilized as a code table to check whether the watermark information is true or false.
Straightforwardly, problems to be solved by the present invention are stated in the Call for Proposals made by the CPTWG/DHSG. In the Call for Proposals, a recording system in the package medium of a moving image and a transmission system in broadcast/communication which satisfy the following specifications are mentioned as below:
Aims: Permitting a content offerer to put a xe2x80x9cDHS watermarkxe2x80x9d indicative of copyright into a digital video medium, for the purposes of identifying the copyright and preventing illegal video recording and reproduction. Here, it is especially intended to prevent a consumer from easily copying the moving image the duplication of which is prohibited.
A digital video equipment in the future will incorporate this standard and will function so as to reject illicit video recording and reproduction. Although this standard is chiefly directed toward DVDs, it is similarly applicable to other video applications (for example: satellite broadcast, cable television, etc.).
The specifications of copy restrictions include three kinds of modes, which are shown in FIG. 1. In addition, the following points are requested as properties/capabilities which the new systems are to possess:
(1) When the moving image is reproduced and displayed, a viewer does not have an offensive feeling.
(2) The copy restriction mode is easily detected by a video recording/reproduction device.
(3) The digital detection of the DHS watermark is easy in various areas (moving image data, compressed data, transmission data, sector data, etc.).
(4) Household video equipment can detect the one-copy mode, and can set a new disk for recording to the no-more-copy mode.
(5) The erroneous rejection of recording is reduced.
(6) The detection rate of the xe2x80x9cDHS watermarkxe2x80x9d is high (for example, at least 50%).
(7) The DHS watermark is held in ordinary video processing which is executed by a viewer.
(8) A license is allowed under reasonable conditions.
(9) The new systems can be exported to and imported from Japan, U.S.A., Europe, and other countries.
(10) The new systems are technologically mature, and can be demonstrated.
(11) Owing to the DHS watermark, the three kinds of copy control modes can be detected continuously anytime.
(12) The DHS watermark can be inserted efficiently by real-time processing.
(13) An encoder/decoder for the DHS watermark is capable of processing at a DVD operation speed.
The present invention is intended to incarnate a more excellent scheme for copyright protection by meeting the above aims and the specifications relevant to the three kinds of copy control modes.
The problem of the prior art concerning electronic-watermark insertion techniques is as explained below. Even in the prior-art schemes, it is aimed at to embed the watermark information with the degradation of the image quality or the tone quality avoided to the utmost, and to make the embedded watermark information difficult of removal to the utmost. These aims, however, incur the problem that considerably complicated processing is necessitated, but that a simple and inexpensive method or the insertion is not existent.
The present invention has for its object to facilitate, for example, the detection of an illicit copy by a watermark insertion scheme of simple construction and low cost in which a visible watermark being visually perceivable in a still image and being visually unperceivable in a moving image is inserted into the moving image. That is, the object of the present invention is to provide a contrivance for protecting copyright in such a way that the indication of the copyright of a moving image, for example, is inserted as a watermark, thereby making it possible to readily expose illegality such as the piracy of the image.
A watermark insertion device according to the present invention is premised on the insertion of watermark information into a moving image composed of a plurality of frames, and it comprises a frame selection unit and a watermark-information insertion unit.
The frame selection unit selects frames into which the watermark information is to be inserted, at irregular intervals from among the plurality of frames. The selection of the frames is effected in such a way that the frames are determined by the use of, for example, random number values which are generated under a controlled condition.
The watermark-information insertion unit inserts the watermark information into the selected frames.
The watermark information to be inserted has, for example, a size which is large enough to visually perceive this watermark information when the moving image is brought to a standstill (into a static image). The number of the frames into which the watermark is to be inserted, is set so that the inserted watermark may not be recognizable in the moving image.
Besides, the watermark insertion device according to the present invention can be so constructed as to further comprise a moving-image storage unit for storing therein the moving image in which the watermark information has been inserted, together with information indicative of the frames in which the watermark has been inserted, or information indicative of the controlled condition which serves for the frame selection unit to select the frames. Also, the moving-image storage unit can be so constructed as to further store therein information indicative of those positions in the frames at which the watermark has been inserted, or a condition for generating random number values which the watermark-information insertion unit uses for determining the insertion positions.
Also, the watermark insertion device according to the present invention can be so constructed as to further comprise a copy-restriction-information insertion unit by which information indicative of a restriction on the copying of the moving image is inserted into an area where control data for the moving image are recorded.
Moreover, the present invention shall cover within its scope a method of generating the moving image in which the watermark is inserted, a storage medium which stores therein the moving image or a program for implementing the functions of the above device by means of a computer, and an apparatus which reproduces the contents of the storage medium.
According to the present invention, a simple and inexpensive scheme for the copyright protection of the moving image can be provided.